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Abstract 

The paper introduces ensembles of accumulate-repeat-accumulate (ARA) codes which asymp- 
totically achieve capacity on the binary erasure channel (BEC) with bounded complexity per 
information bit. It also introduces symmetry properties which play a central role in the con- 
struction of capacity-achieving ensembles for the BEC. The results here improve on the tradeoff 
between performance and complexity provided by the first capacity-achieving ensembles of ir- 
regular repeat-accumulate (IRA) codes with bounded complexity per information bit; these 
IRA ensembles were previously constructed by Pfister, Sason and Urbanke. The superiority of 
ARA codes with moderate to large block length is exemplified by computer simulations which 
compare their performance with those of previously reported capacity-achieving ensembles of 
LDPC and IRA codes. The ARA codes also have the advantage of being systematic. 

Index terms - binary erasure channel (BEC), capacity, complexity, degree distribution 
(d.d.), density evolution (DE), iterative decoding, irregular repeat-accumulate (IRA) 
codes, systematic codes. 



1 Introduction 



Error correcting codes which employ iterative decoding algorithms are now considered 
state of the art in the field of low-complexity coding techniques. By now, there is a 
large collection of families of iteratively decoded codes including low-density parity-check 
(LDPC), turbo, repeat-accumulate and product codes; all of them demonstrate a rather 
small gap (in rate) to capacity with feasible complexity. 

The study of capacity- achieving (c.a.) sequences of LDPC ensembles for the binary 
erasure channel (BEC) was initiated by Luby et al. [b and Shokrollahi |2,. They show 
that it is possible to closely approach the capacity of an erasure channel with a simple 
iterative procedure whose complexity is linear in the block length of the code 13- 
Following these works, Oswald and Shokrollahi presented in [3] a systematic study of c.a. 
degree distributions for sequences of ensembles of LDPC codes whose transmission takes 
place over a BEC. Jin et al. introduced irregular repeat-accumulate (IRA) codes and 
presented a c.a. sequence of systematic IRA (SIRA) ensembles [4J. A new sequence of c.a. 



SIRA codes with lower complexity was also introduced in j^J. All of the aforementioned 
codes have one major drawback; their decoding complexity scales like the log of the 
inverse of the gap (in rate) to capacity, which becomes unbounded as the gap to capacity 
vanishes EI ■ 

In a previous paper [S|, Pfister, Sason and Urbanke presented for the first time two 
sequences of ensembles of non-systematic IRA (NSIRA) codes which asymptotically (as 
their block length goes to infinity) achieve capacity on the BEC with bounded complexity 
per information bit. The new bounded complexity result in jS] is achieved by puncturing 
bits and allowing in this way a sufficient number of state nodes in the Tanner graph 
representing the codes. We note that for fixed complexity, the new codes in jH] eventually 
(for large enough block length) outperform any code proposed so far. However, the 
convergence speed to the ultimate performance limit happens to be quite slow, so for 
small to moderate block lengths, the new codes are not record breaking. 

In this paper, we are interested in constructing c.a. codes on the BEC with bounded 
complexity per information bit which are also systematic codes. We would also like these 
codes to perform well at moderate block lengths and have low error floors. To this end, 
we make use of a new channel coding scheme, called 'Accumulate-Repeat-Accumulate" 
(ARA) codes, which was recently introduced by Abbasfar, Divsalar, and Yao These 
codes are systematic and have both outstanding performance, as exemplified in 011111 HI], 
and a simple linear-time encoding. After defining an appropriate ensemble of irregular 
ARA codes, we construct a number of c.a. degree distributions. Simulations show that 
some of these ensembles perform quite well on the BEC at moderate block lengths. 
Therefore, we expect that irregular ARA codes optimized for general channels might also 
perform well at moderate block lengths. 

Along the way, we study the symmetry of c.a. degree distributions and discover a 
new code structure which we call "Accumulate-LDPC" (ALDPC) codes. We show that 
c.a. degree distributions for this structure can be constructed easily based on the results 
of [HI Theorems 1 and 2]. This fact and structure was also proposed independently by 
Hsu and Anastasopoulos in fF2\ . 

2 Accumulate-Repeat-Accumulate Codes 

In this section, we present our ensemble of ARA codes. Density evolution (DE) anal- 
ysis of this ensemble is presented in the second part of this section using two different 
approaches which lead to the same equation for the fixed points of the iterative message- 
passing decoder (this equation will be called the "DE fixed point equation"). The con- 
nection between these two approaches is used later in this paper to state some symmetry 
properties which serve as an analytical tool for designing various c.a. ensembles for the 
BEC (e.g., ARA, IRA and ALDPC codes). 

2.1 Description of ARA Codes 

ARA codes can be viewed either as interleaved serially concatenated codes (i.e., turbo- 
like codes) or as sparse-graph codes (i.e., LDPC-like codes). From an encoding point 
of view, it is more natural to treat them as interleaved serially concatenated codes (see 
Fig. Q). Since their decoding algorithm is simply belief propagation on the appropriate 
Tanner graph (see Fig. |2J), this leads one to view them also as sparse-graph codes. 
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Figure 1: Block diagram for the systematic ARA ensemble ("Irr." and "SPC" stand for 
"irregular" and "single-parity check", respectively, and II stands for a bit interleaver.) 

Treating these codes as sparse-graph codes also allows one to build large codes by 
"twisting" together many copies of a single small protograph |13| ITi] . In general, this 
approach leads to very good codes and computationally efficient decoders. 

In this work, we consider the ensemble of irregular ARA codes which is the natural 
generalization of irregular IRA codes [U IHj- This ensemble differs slightly from those 
proposed in For this ensemble, we find that DE for the BEC can be computed 

in closed form and that algebraic methods can be used to construct c.a. sequences. 

An irregular ensemble of ARA codes is defined by its degree distribution (d.d.). Nodes 
in the decoding graph will be referred to by the names given in Fig. |21 Let L(x) = 
YliLi LiX 1 be a power series where Li denotes the fraction of "punctured bit" nodes with 
degree i. Similarly, let R(x) = Y^Zi ^ iX% ^ e a P ower series where Ri denotes the fraction 
of "parity-check 2" nodes with degree i. In both cases, the degree refers only to the 
edges connecting the "punctured bit" nodes to the "parity-check 2" nodes. Similarly, let 
= Y^iLi \ xl1 an d p(x) = YltLi Pi x% ~ X form the d.d. pair from the edge perspective 
where \ and pi designate the fraction of the edges which are connected to "punctured bit" 
nodes and "parity-check 2" nodes with degree i, respectively. We also assume that the 
permutation in Fig. ^ is chosen uniformly at random from the set of all permutations. 
The pair of degree distributions of an ARA ensemble is given by (A, p). 

It is easy to show the following connections between the d.d. pairs w.r.t. the nodes 
and the edges in the graph: 

L'(x) R'(x) 
A(X) = W) ' P{X) = R{1) (1) 

or equivalently, since L(0) = R(0) = 0, then 

, f n x A(t) dt , ^ pit) dt 

Jo X (t) dt J Q pit) dt 



2.2 Density Evolution of ARA Ensembles 

We consider here the asymptotic analysis of ensembles of ARA codes. We assume that 
the codes are transmitted over a BEC with erasure probability p and decoded with an 
iterative message-passing decoder. 

A single decoding iteration consists of six smaller steps which are performed on the 
Tanner graph of Fig. |21 Messages are first passed downward from the "systematic bit" 
nodes through each layer to the "code bit" nodes. Then, messages are passed back 
upwards from the "code bit" nodes though each layer to the "systematic bit" nodes. 
Let / designate the iteration number. Referring to Fig. |2j let Xq and x<p designate the 
probabilities of an erasure message from the "parity-check 1" nodes to the "punctured bit" 
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Figure 2: Tanner graph for the ARA ensemble. 



nodes and vice- versa, let x\ and x\ be the probabilities of an erasure message from the 
"punctured bit" nodes to the "parity-check 2" nodes and vice versa, and finally, let 
Xjr and 4 be the probabilities of an erasure message from the "parity-check 2" nodes 
to "code bit" nodes and vice versa. As the block length goes to infinity, the cycle-free 
condition holds with probability 1 and this implies that the messages become statistically 
independent with probability 1. Under this assumption, we obtain the following DE 
equations (from Fig. |2J) for the message-passing iterative decoder: 

= i-(i-4°)(i-p) 

= (4°) 2 a(4°) 

= 1 - R{1 - x?) (1 - 4°) 

(0 

= px 2 

= 1 - (1 - of) 2 p(l - X?) 
= 4° L{xf) 

A fixed point is implied by 

lim xp = Xi % = 1, . . . , 5. 

Algebra shows that if X\ = x, then we obtain the following equation for the fixed points 
of the iterative decoder: 

p 2 a(i-(i=^))%(i-^)) 

For ensembles of ARA codes whose transmission takes place over a BEC, the DE 
fixed point equation © can be also derived using a graph reduction approach. 

We start by noting that any "code bit" node whose value is not erased by the BEC 
can be removed from the graph by merging its value into its two "parity-check 2" nodes. 
On the other hand, when the value of a "code bit" node is erased, one can merge the two 
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"parity-check 2" nodes which are connected to it (by summing the equations) and then 
remove the "code bit" node from the graph. This merging of the two "parity-check 2" 
nodes causes their degrees to be summed. Now, we consider the degree distribution (d.d.) 
of a single "parity-check 2" node in the reduced graph. This can be visualized as working 
from left to right in the graph, and assuming the value of the previous "code bit" node 
was known. The probability that there are k erasures before the next observed "code bit" 
is given by p k (l — p). The graph reduction associated with this event causes the degrees 
of k + 1 "parity-check 2" nodes (from the d.d. R(x)) to be summed. Therefore, the new 
d.d. of the "parity-check 2" nodes after the graph reduction is given by 

%x) = ±Ai-p)m M = { \z p ™ . (4) 

fc=0 " ^ ' 

A similar graph reduction can be also performed on the "systematic bit" nodes in 
Fig. El Since degree 1 bit nodes (e.g., the "systematic bit" nodes in Fig. EJ) only provide 
channel information, erasures make them worthless. So they can be removed along 
with their parity-checks (i.e., the "parity-check 1" nodes in Fig. EJ) without affecting the 
decoder. On the other hand, whenever the value of a "systematic bit" node is observed 
(assume the value is zero w.o.l.o.g.), it can be removed leaving a degree 2 parity-check. Of 
course, degree 2 parity-checks imply equality and allow the connected "punctured bit" 
nodes to be merged (effectively summing their degrees). This gives a nice symmetry 
between the information bits and parity bits. Now, we consider the d.d. of a single 
"punctured bit" node in the reduced graph. This can be seen as working from left to 
right in the graph, and assuming the value of the previous "systematic bit" node was 
erased. The probability of the event where the values of k "systematic bit" nodes are 
observed and the value of the next "systematic bit" node is erased by the channel is given 
by (1 —p) k p. The graph reduction associated with this event causes the degrees of k + 1 
"punctured bit" nodes (from the d.d. L(x)) to be summed. Therefore, the new d.d. of 
the "punctured bit" nodes after graph reduction is given by 

K^gl-riW ,.^,) . (5) 

After the graph reduction, we are left with a standard LDPC code with new edge- 
perspective degree distributions given by 

X(x) = iM = (6) 

L'(l) (1 - (1 - p)L(x)) 2 

p(x) = R y ] - a-^w m 

R'{1) (l-pR(x)) 2 ' 

After the aforementioned graph reduction, all the "systematic bit" nodes and "code bit" 
nodes are removed. Therefore the residual LDPC code effectively sees a BEC whose 
erasure probability is 1, and the DE fixed point equation is given by 

\{l-p(l-x))=x. (8) 

Based on (0) and (J7J), the last equation is equivalent to ©• We note that although A 
and p which are given in © and (J7J), depend on the erasure probability of the BEC (p), 
for simplicity of notation, we do not write this dependency explicitly in our notation. 
However, in Section |31 when discussing symmetry properties and replacing p by 1 — p, 
the erasure probability is written explicitly in these tilted degree distributions. 



3 Symmetry Properties of Capacity- Achieving Codes 



In this section, we discuss the symmetry between the bit and check degree distributions 
of c.a. ensembles for the BEC. First, we describe this relationship for LDPC codes, and 
then we extend it to ARA codes. The extension is based on analyzing the decoding of 
ARA codes in terms of graph reduction and the DE analysis of LDPC codes. 



3.1 Symmetry Properties of Capacity- Achieving LDPC Codes 

The relationship between the bit d.d. and check d.d. of c.a. ensembles of LDPC codes 
can be expressed in a number of ways. Starting with the DE fixed point equation 

pX(l - p(l - x)) = x (9) 

where p designates the erasure probability of the BEC, we see that picking either the d.d. 
A or p determines the other d.d. exactly. In this section, we make this notion precise 
and use it to expose some of the symmetries of c.a. LDPC codes. 

A few definitions are needed to discuss things properly. Following the notation in 
let V be the set of d.d. functions (i.e., functions / with non-negative power series 
expansions around zero which satisfy /(0) = and /(l) = 1); this set is defined by 

{oo 
/ : f(x) = fkJ, * G [0, 1], f k > 0, /(0) = 0, f(l) = 1 
fc=i 

Let T be an operator which transforms invertible functions / : [0, 1] — > [0, 1] according 
to the rule 

r/(*) 4 1 _ _ x) 

where / _1 is the inverse function of /. The function Tf is well-defined on [0, 1] for any 
function / which is strictly monotonic on this interval, and therefore for any function in 
V. We will say that two d.d. functions / and g are matched if Tf = g (since T 2 f = f, 
the equality Tf = g implies that Tg = f). Finally, let A be the set of all functions 
f eV such that Tf G V, i.e., 



A±{f:feV,Tfev}. 



The connection with LDPC codes is that finding some / G A is typically the first step 
towards proving that (/, Tf) is a c.a. d.d. pair. Truncation and normalization issues 
which depend on the erasure probability of the BEC must also be considered. When 
p — 1, many of these issues disappear, so we denote the set of d.d. pairs which satisfy 
© by 

Cldpc = {(X,p)ePxP\ X(l-p(l-x))=x} 
= {(A,p)| XeA, P = Tx}. 
The symmetry property of c.a. LDPC codes (with rate 0) asserts that 

(A, p) G C LD pc < symmetry . ( p> A) g C LDPC . (10) 

One can prove this result by transforming Q whenp = 1. First, we let x = 1 — p _1 (l — y), 
which gives X(y) = 1 — p _1 (l — y), then we rewrite this expression as p(l — X(y)) = l — y 
and finally, let y — 1 — z to get p(l — A(l — z)j = z. Comparing this with the DE fixed 
point equation Q when p — 1 shows the symmetry between A and p. 



3.2 Symmetry Properties of ARA Codes 



The decoding of an ARA code can be broken into two stages. The first stage transforms 
the ARA code into an equivalent LDPC code via graph reduction, and the second stage 
decodes the LDPC code. This allows us to describe the symmetry property of c.a. ARA 
codes in terms of the symmetry property of c.a. LDPC codes. For / G V, let us define 

Y _ pJq /W d M 



Jo ?W* J 

One can write the d.d. pair (A, p) after graph reduction by combining (J2J), © and (J7J) 
which gives 

A = Ai_ p , p = p p . 

This allows graph reduction to be interpreted as a mapping £ara from an ARA d.d. pair 
to an LDPC d.d. pair which can be expressed as 

C/ARA „ 

(A,p) 5 (Ai_p,p p ). 

The inverse of the graph reduction mapping is represented by a dashed arrow because 
this inverse mapping, while always well-defined, does not necessarily preserve the non- 
negativity of d.d. functions. 

Referring to ensembles of ARA codes, the set of d.d. pairs which satisfy the DE fixed 
point equation (J3J) is given by 

Cara(p) = {(X,p)eVxV\ X^ p (l - p p (l - x)) = x } 

where the equivalence to (J3J) follows from (JBJ), (J7J) and (fTTj) . 

The symmetry between the bit and check degree distributions of a c.a. ARA ensemble 
follows from the symmetry relationship in (jl(Jj) . and the equivalence between a d.d. pair 
(A, p) for ARA codes and the d.d. pair (Ai_ p , p p ) for LDPC codes. 

The complete symmetry relationship is therefore given in the following diagram: 

ARA symmetry 
(A, p) G Cara(p) " - (p, A) G C A ra(1 - p) 



QarA 



QarA 



~ _ LDPC symmetry ~ 
(Ai-p, Pp) G Cldpc " " [Pp, Ai-p) G Cldpc 

The inverse of the graph reduction mapping is represented by the dashed arrow because 
this inverse transformation is only valid if it is known ahead of time that the power series 
expansions of A and p are non-negative. It turns out that this symmetry is very useful 
in order to generate new d.d. pairs which satisfy the DE equality in (jSJ). An alternative 
way to show this symmetry explicitly is rewriting (JSJ) 

Ai_ p (l - pp(x)) = x 

and using the symmetry property (fTUJ) for LDPC codes to rewrite it as 

p p (l - Ai_ p (x)) = 1 - x. 



From (JBJ) and (J7j) , the expansion of the last equation gives 



(i-p) 2 pfi- , p2x{x) ,0 

V, (l-(l-p)L(x)) / 

1 1 - - p2A(a:) a n 

y ' V (l-(l- P )L(x)) V / 



1 — X. 



(12) 



Since the swapping <-> R(x), X(x) «-> p(x), p «-> 1 — p, and x <-> 1 — x maps this 
equation back to ©, then we can take any d.d. pair (A, p) which satisfies © for p = p* 
and swap A with p (and hence, L and R are also swapped) to get a new d.d. pair which 
satisfies (fT2|) for p = 1 — p* (equations (jHJ) and (fT2|) should be satisfied for all x G [0, 1], 
so switching between x and 1 — x has no relevance). 

3.3 Symmetry Properties of NSIRA Codes 

Now, we consider the graph reduction process and symmetry properties of non-systematic 
irregular repeat-accumulate (NSIRA) codes (for preliminary material on NSIRA codes, 
the reader is referred to [HI Section 2]). In this respect, we introduce a new ensemble of 
codes which we call 'Accumulate-LDPC" (ALDPC) codes. These codes are the natural 
image of NSIRA codes under the symmetry transformation. In fact, this ensemble was 
discovered by applying the symmetry transformation to previously known c.a. code 
ensembles. Their decoding graph can be constructed from the ARA decoding graph (see 
Fig. |2J) by removing bottom accumulate structure. 

Since an NSIRA code has no accumulate structure attached to the "punctured bit" 
nodes, the graph reduction process affects only the d.d. of the "parity-check 2" nodes. 
Therefore, graph reduction acts as a mapping (?nsira from the NSIRA d.d. pair (A, p) to 
the LDPC d.d. pair (A, p p ). This yields that for ensembles of NSIRA codes, the set of 
d.d. pairs which satisfy the DE fixed point equation is given by 



An ALDPC code has no accumulate structure attached to the "parity-check 2" nodes, 
and therefore the graph reduction process only affects the d.d. of the "punctured bit" 
nodes. Hence, graph reduction acts as a mapping {?aldpc from the ALDPC d.d. pair 
(A, p) to the LDPC d.d. pair (Ai_ p , p). For ALDPC ensembles, the set of d.d. pairs 
which satisfy the DE fixed point equation is therefore given by 



The symmetry between NSIRA and ALDPC ensembles follows from the symmetry 
relationship in (JTUJ), the equivalence between a d.d. pair (A, p) for NSIRA codes and the 
d.d. pair (A, p p ) for LDPC codes, and the relationship between a d.d. pair (A, p) for 
ALDPC codes and the d.d. pair (A]_ p ,p). 

The symmetry relationship is therefore given in the following diagram. 





(A,p) G C N sira(p) 



symmetry 



(p, A) G C A ldpc(1 -p) 



{?NSIRA 



{?ALDPC 



(A, Pp) G Cldpc 



LDPC symmetry 



[Pp, A) G Cldpc 



As before, the inverse of each graph reduction mapping is represented by a dashed 
arrow because this inverse transformation is only valid if it is known ahead of time that 
the power series expansions of A and p are non-negative. 

3.4 Connections with Forney's Transform 

In Forney introduces a graph transformation which maps the factor graph of any 
group code to the factor graph of the dual group code. For factor graphs of binary 
linear codes which only have equality and parity constraints (i.e., no trellis constraints), 
this operation is equivalent to swapping equality and parity constraints (e.g., bit nodes 
and check nodes). Forney's approach represents observations by half-edges, and these 
remain attached to the original node even though the nature of that node has changed. 
For example, Forney's transform maps an LDPC code with parity-check matrix H to a 
low-density generator-matrix (LDGM) code with generator matrix H and the half-edges 
attached to the bit nodes of the LDPC code are attached to the parity-check nodes of 
the LDGM code. 

Using Forney's transform, we see that the swapping of A and p described by our 
symmetry mappings actually transforms the original code ensemble into the dual code 
ensemble. Let the design rate of the original ensemble be R, then the design rate of the 
dual ensemble is 1 — R. This means that if we want to have any chance of achieving 
capacity, we must also map the channel erasure probability p to 1 — p. Therefore, our 
symmetry relationships show that ARA, NSIRA, and ALDPC ensembles which are c.a. 
on BEC under iterative decoding also have dual ensembles which are c.a. on the BEC 
under iterative decoding. 1 

Finally, we note that the basic structure of ARA codes is preserved under Forney's 
transform. In particular, this means that we can construct self-dual ARA codes (i.e., rate 
|) by choosing the square connection matrix between the "punctured bit" nodes and the 
"parity-check 2" nodes to be symmetric. 

4 Bit-Regular and Check- Regular Capacity- Achieving 
Ensembles with Bounded Complexity for the BEC 

This section gives explicit constructions of c.a. ARA ensembles for the BEC, where these 
ensembles are either bit-regular or check-regular. As will be observed, these ensembles 
possess bounded complexity (per information bit) as the gap to capacity vanishes. 

The symmetry property in Section 13.21 allows one for example to design an ensemble 
of high rate ARA codes, and get automatically (by switching between the pair of degree 
distributions) a new ensemble of ARA codes which is suited for low rate applications. 
We will rely on this symmetry property in Section 14.21 when we transform a bit-regular 
ARA ensemble designed for a BEC with erasure probability p € (0,p*] into a check- 
regular ensemble designed for p £ [1 — p*, 1). We also rely on the fact that the method 
in Section 14.11 for computing the function R given the function L can be easily inverted 
using the symmetry property. This means that given an algorithm to solve for R(x) in 

1 To be precise, we actually need to consider sequences on ensembles which are c.a. and relate them 
to sequences of dual ensembles. This distinction is rather cumbersome and does not cause problems in 
this case. 



terms of L(x) for a certain p , the inverse algorithm which solves L(x) in terms of R(x) 
is exactly the same, except that p is replaced by 1 — p . 



4.1 Solving for R(x) in terms of L(x) 

Given L(x), we start with the calculation of X(x) = xrjyy- Then X(x) is calculated from 
(JHJ), and p(x) = 1 — A _1 (l — x) is calculated from (JHJ). Further algebra gives 

p(x) = — m — , Q(x] a iiem . (13) 

As long as we have p(l) = 1, then evaluating (J13j) at x — 1 gives p(l) = 1. Therefore, 
there is no need to truncate the power series of p. As we noted above, a very similar 
approach can be applied to solve for L{x) in terms of R(x); due to the symmetry property, 
one can simply apply the above procedure to a parity-check d.d. R(x) with an erasure 
probability of 1 — p. 



4.2 Capacity- Achieving ARA Ensembles 

The relationship between bit-regular and check- regular c.a. ensembles of ARA codes 
follows from the symmetry properties presented in Section 13.21 so we choose to focus on 
a bit-regular ARA ensemble. Let X(x) = x 2 , so L(x) = x 3 , and from © 

X(x) = - p2x2 



(1 — p)x 3 ) 
Based on (jHJ), we get 

p~ x (x) = l-A(l-x) 

p 2 (l — x) 



1 - 



(i-(i- p )(i- x yy 

This is exactly [8, Eq. (39)] with p replaced by 1 — p and the d.d. p switched with the 
d.d. A. Therefore, we obtain from Theorem 2] that the tilted d.d. p gets the form 



2(1 —p)(l — x) sin | arcsin 



2<m,l LwcmL/ 27(1- P )(1-X)I 



3 "^"^ I W A p i 
P{X) = 1 + ^ ^ 3 — • < I J ) 

This allows one to write the d.d. p compactly using ()13j) . It was verified numerically that 
for p < 0.384, the first 300 coefficients in the power series expansion of R(x) are positive. 
It also holds in general that any d.d. pair satisfying (jSJ) has a design rate equal to the 
capacity of the BEC. Therefore, it appears that the d.d. pair above characterizes a c.a. 
ensemble of bit-regular ARA codes over the BEC; the capacity of the BEC is achieved 
with bounded complexity for rates greater than 0.616. Using the symmetry between A 
and p (see Section |HJ), this also implies that for rates less than 0.384, the ensemble of 
check-regular ARA codes with R(x) = x 3 achieves capacity over the BEC with bounded 
complexity. We note that the convergence speed of the d.d. for the parity-check nodes is 
relatively fast. As an example, for p = 0.3, the fraction of check nodes with degree less 
than 32 is equal to 0.968. 



4.3 Capacity- Achieving ALDPC Ensembles 



Using the symmetry relationship between NSIRA and ALDPC ensembles from Sec- 
tion we find that we already have from JHJ Theorems 1 and 2] two c.a. ensembles of 
ALDPC codes. These ensembles are based on the bit-regular and check-regular NSIRA 
ensembles of [HJ. This was also observed independently by Hsu and Anastasopoulos |12j . 

Using symmetry, the check-regular NSIRA ensemble gives a bit-regular ALDPC en- 
semble which provably achieves capacity with bounded complexity for p G [0.05,1). 
Random puncturing can be viewed as increasing the effective erasure rate of the chan- 
nel, and therefore puncturing extends this range to (0,1). Likewise, using symmetry, 
the bit-regular NSIRA ensemble gives a check-regular ALDPC ensemble which provably 
achieves capacity with bounded complexity for p G , l) - Again, random puncturing 
can be used to extend the valid range to (0, 1). 

5 Capacity- Achieving Ensembles with Bounded Com- 
plexity: Constructions Based on LDPC Codes 

In this section, we introduce another way of constructing c.a. ensembles of ARA codes 
for the BEC. Rather then solving for the function R in terms of the function L (as in 
Section 14. lj) or doing the inverse via the symmetry property, we consider here another 
natural way of searching for c.a. degree distributions. We start by choosing a candidate 
d.d. pair (A,p) which satisfies equation (JHJ) and test to see if it can be used to construct 
an ensemble of c.a. ARA codes. The testing process starts by mapping the tilted pair 
(A, p) back to (A, p) via (JHJ) and 0, and then testing the non-negativity of the resulting 
power series of A and p. 

Following the notation in Section it enables one to rewrite (JHJ) as p = TX (so 
the tilted degree distributions A and p are matched), and gives a compact description 
of capacity-achieving d.d. pairs of LDPC codes. We note that since T 2 f = f for an 
arbitrary function / which has an inverse, then / G A if and only if Tf G A. Based on 
(JHJ), we obtain that we need to choose the tilted d.d. so that A G V and also TX G V, 
i.e., we need that the d.d. A (or p) both belong to the set A. The reader is referred to 
jHJ Lemma 1] which considers basic properties of the set A and the transformation T. 

So far, by choosing A G A (or p G -4), we only know that both tilted d.d. have 
non-negative power series expansions. This property does not ensure that both of the 
original (i.e., non-tilted) d.d. A and p also have non-negative power series expansions. 
Calculation of A and p from the tilted d.d. A and p is not straightforward since both 
equations involve the d.d. L and R which are the normalized integrals of the unknown 
A and p. In order to overcome this difficulty in solving the two integral equations, we 
suggest calculating the tilted d.d. pair w.r.t. the nodes of the graph using 



The original d.d. pair w.r.t. the nodes (i.e., the original d.d. pair before the graph 
reduction) can be calculated from Eqs. (JIJ) and (JHJ)- We obtain that 



L(x) 



fi\{t) dt 



R(x) 



Jo P(t) dt 
f*p(t)dt 



(15) 



L(x) 



R(x) 



R(x) 



(16) 



p+ (1 -p)L(x) 



1 — p + pR(x) 



and then use equation to find (A, p). The critical issue here is to verify whether the 
functions L and R have non-negative power series expansions. 



5.1 Capacity- Achieving ARA Ensembles 

It is easy to verify that the function 

/W = T^. °< &<1 ( 17 ) 

belongs to the set A and also Tf = /; in the case where Tf = f, the function / is said 
to be matched to itself. Therefore, based on (jHJ), we examine here whether the choice 
X(x) = p(x) = can be transformed into an ensemble of ARA codes whose degree 

distributions have non-negative power series expansions. From (|T3j) and (|T^|) . we get 

~ . ~ . bx + ln(l — bx) , . 

H*) = R{*)= 6 + ln | 1 _ 6) J («) 

and LM = bx + HI - bx) 

K) p[6 + ln(l-6)] + (l-p)[6a: + ln(l-6x)] 1 ' 

R(x ) = bx + ln ( X ~ hx ) (20) 

K > (l-p)[b + ln(l-b)]+p[bx + ln(l-bx)Y V } 

The asymptotic behavior of the d.d. pairs w.r.t. the nodes and the edges is given by 

/ b k \ ( b k 

Lk, Rk = O 2 , Afe, pk = O 



We believe the performance advantage of this ensemble over other c.a. ensembles is 
mainly due to the exponential decay of the d.d. coefficients. 

It has been observed empirically, that the power series expansions of both R and L 
are non-negative if and only if p satisfies the inequality 

1 1 

f= <p<l = 21 



9 

and 



(6 + ln(l-6)) 1 - (b + ln(l - b)) 

6e[6*,l), b* ^Wi-e- 2 ^ 1 ) + 1^ 0.9304 



where W designates the Lambert W-function (i.e., w = W(x) is the solution to the 
equation we™ = x, which is a real number for x > —-)■ These conditions follow since 
it was observed numerically that the strongest conditions for the non-negativity of the 
power series expansions of L and R are implied by the coefficients L$ and Rq, respectively. 

The encoding and decoding complexities per information bit of the considered c.a. 
ensembles of ARA codes for the BEC are bounded and given by 

3 — p b 2 p 
XE ' XD = Y^~p ~ (1 -&)[& + ln(l -&)] ' 

For fixed p, the complexity is a monotonic increasing function of b (which becomes 
unbounded as b — > 1~). To minimize the encoding/decoding complexity, we wish to 
choose the smallest value of b in the interval (0, 1) so that all the coefficients in the power 
series expansions of the d.d. pair L and R are non-negative. We therefore need to choose 
the smallest value of b (0 < b < 1) which satisfies the condition in (J21j) . Since p and 1—p 
imply the same value of b in (|21j). the required value of b is given by 

^W,-e-) + l, 

12 \min(p, 1 — p) 



5.2 Capacity- Achieving NSIRA Ensembles 



In this section, we construct ensembles of NSIRA codes using LDPC codes whose degree 
distributions from the edge perspective are matched. We apply here the concept of DE 
via graph reduction to ensembles of uniformly interleaved NSIRA codes. In this case, the 
graph reduction only applies to the "parity-check 2" nodes (see Fig. |2J). This is because 
the upper part of Fig. |2] does not exist in the Tanner graph of NSIRA codes (i.e., the 
"punctured bit" nodes in this figure are the "information bit" nodes in the graph of 
NSIRA codes). Based on graph reduction, we obtain that L = L for ensembles of NSIRA 
codes, while the functions R and R satisfy the equality in (|16|). In a similar manner, the 
equality A = A holds for NSIRA ensembles while equality (J2J) is satisfied for the degree 
distributions of the parity-checks from the edge perspective. We note that from (JJJ) and 
(JEJ), the fixed point of the DE equations for NSIRA ensembles is given by 

V (\-pR(\-x)Y) 

This equation coincides with Eq. (6)] (where x is replaced by x). 

For the construction of ensembles of NSIRA codes using LDPC codes whose degree 
distributions from the edge perspective are both matched to themselves, we rely as a 
starting point on the function / in (|17|) which forms a d.d. which is matched to itself, 
and set X(x) = p(x) = f° r < 6 < 1, similarly to Section IHTT1 For the considered 

ensemble of NSIRA codes, the d.d. L(x) is then equal to L(x) in (fI5]l. i.e., 

bx + ln(l — bx) 
[X) = 6 + ln(l-6) • 

From this, we see that there are no degree-1 "information bit" nodes, and that the fraction 
of "information bit" nodes with degree i is given by 

U 1 ■ _ o Q 

6 + ln(l-6) ' *" 2 ' 3 '---- 

The non-negativity of the sequence {Li} holds when < b < 1 (so b + ln(l — b) < 0). For 
the NSIRA ensemble considered in this example, there is no requirement on the erasure 
probability p for keeping the power series expansion of the d.d. L to be non-negative. It 
has been empirically observed that the following condition on p needs to be satisfied so 
that the power series expansion of the d.d. R will be non-negative: 

p<l . (22) 

1_ 13^61 [ 6 + i n (i_ 6 )] 

By comparing it to the parallel requirement for the ARA ensemble, as given in (j21)l . 
one observes that ()22|) requires a weaker condition on p which is only the upper bound 
on p in (j21j) . As mentioned above, the d.d. R is the same as for the ARA ensemble in 
Section lo~Tl The encoding and decoding complexities of this ensemble are equal and have 
the form 

_2 9 

Xe-Xd- ~ (i -b)[b + ln(l - b)] ' 

This gives an explicit construction of NSIRA ensembles from LDPC codes whose 
degree distributions from the edge perspective are matched to themselves. In general, we 
find by computer simulations for finite-length codes over the BEC that ARA codes have 
the best performance. 
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Figure 3: Simulations are shown for the ensembles of ARA and NSIRA codes in Sec- 
tions l5~Tl and 15.21 and right-regular LDPC codes [2]. The plots refer to block lengths of 
n = 8192 and 65536 bits (see upper and lower plots, respectively) and a design rate of 0.5 
bits per symbol. Since the ensemble averaged performance is simulated, high-rate outer 
codes (rates 8179/8192 and 65520/65536, respectively) are used to lower the error floor 
due to small stopping sets. These outer codes are chosen uniformly at random from the 
ensemble of the binary linear block codes and their rate loss is neglected. 



